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Abstract 


Confrontation with difficult manipulation tasks in hostile environments such as space, has 
led to the development of means to transport the human's senses, skills and cognition to the remote 
site. We examined the use of advanced Telerobotics to achieve this goal. A novel and universal 
hand controller based on a fully parallel mechanical architecture is discussed. The design and 
implementation of this 6 DOF force reflecting joystick is shown in relationship to the general 
philosophy of achieving telepresence in a man-machine system. 

1. Introduction 


This paper describes the work undertaken at the University of Texas at Austin to construct 
and implement a force reflecting universal hand controller in a microprocessor driven testbed with 
an industrial robot as discussed in [1]. The Texas 9-string kinesthetic joystick has been interfaced 
to a robotic manipulator and a microprocessor to realize a prototype telerobotic system. The system 
is a generalization of the industrial bilateral master-slave teleoperator. The man-machine interface is 
universal and therefore capable of positioning and orienting any 6 DOF manipulator once the 
suitable transformation changes are made in the controlling software. The 9 string kinesthetic 
joystick represents the extension of force reflection to the original 9 string unilateral joystick 
developed by Tesar and Lipkin as discussed in [2]. The design of the joystick has been based on 
maximizing its capability to convey telepresence through a novel parallel architecture which is 
actuated in antagonism. The Texas telerobotics system represents an experimental test facility for 
research into the engineering and human factors issues of man-machine interface. In the table 
below the design goals of the project are given. 


1) decoupled interface: 

2) motion projection: 

3) variable control point: 

4 ) accommodation: 

5) coordinated control: 

6) motion filtering: 

7) positional scaling: 

8) indexing: 

9) reorientation: 


System Functional Attributes 

no geometric similarity required between master and slave 
projection of commanded motion 
electronic control point selection 
manipulator motion is altered by the end-effector 
operator directly controls end-effector motion 
jitters and jerks in input motion removed 
variable positional gain between interface and manipulator 
controller repositioning 
compensation for operator perspective 
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2. Man-machine interface 

Because of man's need to have control or an effective presence to do manipulation in 
remote or hostile environments the teleoperator system (TOS) has been developed. These TOS 
have given man the ability to extend his strength and dexterity along with his intelligence into the 
remote site. Historically, the TOS consisted of two manipulator arms which were geometrically 
identical. One arm, called the master served as the control input device positioned by the operator. 
The other arm, called the slave, could if servoed sense and feedback its load state to the master 
arm. 


The importance of TOS is in its capacity to extend to a human operator the remote control 
of the full 6 degrees-of-freedom (DOF) of rigid body motion through the positioning of one hand. 
In an advanced TOS the man is only one component along with the computational base, remote 
manipulator, display facility, sensory hardware, communication system, and the control input 
device. In figure 1 below, a schematic of such a generic system is shown with the arrows 
indicating the flow of communication signals. 



Time and Distance 


Figure 1: Advanced Generic Teleoperator System 

An important characteristic of a TOS is the degree to which the operator is made to feel he 
is at the remote site actually performing the manipulation task. This illusive design feature is termed 
'telepresence'. Studies conducted on advanced TOS indicate the need for force feedback to the 
operator from the remote manipulator. Thus, the TOS must condense the vast quantity of data that 
is echoed back from the environment of the manipulator into a form favorable to human perception 
and interpretation. Simultaneously, the human's limited output must contain sufficient information 
for unambiguous interpretation by the computational base. The result is a very comprehensive 
control input device. To achieve the most effective relationship between the operator and the 
manipulator, the control input device or man-machine interface should be effectively transparent to 
the information flowing through it. 

In order to function efficiently, the TOS used in general, unstructured tasks will require 
specific slave manipulator geometries which may vary greatly in size. The man-machine interface 
must be constructed with respect to its utility as a control input and kinesthetic feedback device. 
The intersection of these two demands dictates the need for a universal manual controller. The 
universal controller is one which is fully software driven and requires a computational base or 
machine intelligence to drive two geometrically dissimilar manipulator arms. The inclusion of a 
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machine intelligence into the TOS, frees the man-machine interface designer from the restrictions of 
kinematic replication and limited control in the development of a generalized master-slave TOS. 

As indicated earlier, the state-of-the-art in TOS are replica (geometrically identical) 
master/slave systems, essentially a 30 year old technology that will not be adequate in difficult task 
environments such as orbital and interplanetary space. These systems lack transparency in the 
bilateral flow of communication that causes the operator to be between 2 and 20 times as slow as his 
functioning without a TOS, and generally precludes altogether complex tasks. 

Our belief is that the most effective TOS will incorporate a universal man-machine interface 
optimized in its design to the relevant human factors involved in order to achieve telepresence. As a 
result the man-machine interface will have a geometry distinct and decoupled from that of the 
manipulator being controlled. The interface will then require a computer to augment the human 
intelligence as a computational base performing the needed geometric transformations between the 
man-machine interface and the manipulator. The form of this idealized controller is a universal 
bilateral position controller. 

Advances in the last 25 years have also led to the development of programmable and 
autonomous manipulator arms called robots. A recent result has been their combination with TOS 
into a hybrid form of system called 'telerobotics'. The resulting system can be defined as a robotic 
system which in addition to its usual autonomous modes of operation can take control information 
directly from a human operator through a man-machine interface thus becoming teleoperated; or 
from a higher, supervisory level of executive control, thereby acting in a semi-autonomous 
manner. By making the universal master controller bilateral, the resulting system becomes 
conceptually two dissimilar cooperating robots, software coupled and running in real time. 

3. Design and Analysis 

In the past, problems with man-machine interfaces have included their inertia, backlash in 
their drive trains, friction, and limited or non-ergonomic motion capability. Transparency in the 
flow of communication signals requires that the inertial dynamics and friction effects of the man- 
machine interface be well below the intended feedback level in order to avoid operator confusion 
between signal and noise. In this project an isotropic controller has been sought with a constant 
(but programmable) joystick-to-end-effector position mapping and end-effector-to-joystick load 
state mapping. 

Usually in a TOS or telerobotic system the hand controller is designed around the robot or 
remote manipulator arm which is designed around the tasks it is meant to perform. In contrast, our 
goal has been to design a universal man-machine interface around the human operator and use the 
necessary geometric software transformations in a computer. The forerunner of this project is the 
work of Lipkin (1983) in the design and construction of a unilateral 9-string joystick [3]. This 
work had then been followed by the initial configuration study for a 9-string bilateral joystick 
finished in 1986 by Agronin [4], 

Therefore, we designed the joystick to minimize the interface dynamics and maximize the 
force feedback capability. In order to reduce the inertia associated with each of the air cylinders, an 
optimization has been performed and the point near the air cylinders closest to its moving centroid 
has been chosen as the pivot point in order to minimize the inertia. The air cylinder is connected to 
a universal joint by brackets. An additional benefit to this choice is that the air cylinder is supported 
near its center of gravity and most of the weight of the air cylinders is off loaded. 

In order to maximize the force feedback a geometric optimization of the geometry of the 9- 
string joystick has been performed. The optimization has been used to design for the largest fixed 
minimum of maximum force feedback for use in the open loop control of the feedback signal. In 
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order to assure an isotropic nature to the force feedback, the smallest maximum force that can be 
generated anywhere in the joystick workspace in any direction is the limiting factor. In order to 
maximize that quantity an analysis has been completed which relates the minimum force maximas 
to the geometry of the base triangles (where the cables emanate from the supporting structure of the 
joystick frame), the distance from the base triangle to the center point of the joystick workspace, 
and the air cylinder constant force. 

The analysis approach (detailed in [1]) finds the algebraic rule that expresses the minimum 
of maximum force in a plane and then rotates the plane about the workspace center point and the 
line of action of air cylinder constant force. The calculus of minimization in one variable has then 
been used to find the minimum of force maximums. The technique has then been developed into a 
computer program which uses a global search technique to scan the joystick workspace. The 
program is interactive and the user inputed design factors in an adaptive fashion. The force 
feedback has been found to degrade as the volume of workspace increases. The final design 
chose an equal angle of 34.5 degrees between the strings and air cylinder shafts at the workspace 
center point, a pivotal offset of 0.0 inches, and an equilateral base triangle dimension of 20.83 
inches. 


As indicated earlier, the inertial dynamics of a kinesthetic controller is an important 
description of its quality of transparency or fidelity (signal-to-noise ratio). Therefore, a method of 
dynamic simulation has been performed based on the method of Tesar and Freeman [5], The 
method uses dynamic equations based on influence coefficients which separate the purely position 
dependent functions from those which are time dependent (velocity, acceleration, etc.). An 
interactive program has been written and run simulating the Texas 9-string joystick undergoing a 
variety of path motions under representative velocities and accelerations. 

The results of the simulation can only be summarized here (see [1]); but showed the inertial 
forces to remain at below 3% of the intended force feedback level even when the velocities and 
accelerations of the handgrip were at their peak representative values. The relatively small level of 
inertial force disturbance is to be expected as this along with high stiffness are representative 
properties of parallel mechanisms. 

The choice of joystick working volume or that workspace the T-shaped handgrip can be 
moved within has been based on information found in the literature on other manual controllers 
which showed no debilitation using workspaces in the vicinity of a 12 inch cube [2]. Since an 
initial decision to use 18 inch stroke air cylinders as the compressive actuators had been made, the 
resulting approximate workspace of an 18 inch sphere has been deemed acceptable. 


JOYSTICK DESIGN OBJECTIVES 


joystick workspace 
dexterous workspace 
incremental translation 
orientational range 
incremental orientation 
force feedback range 
torque feedback range 


18 inch diameter sphere 
10 inch diameter sphere 
. 1 3 inch 

1 80 degrees (3 axis) 

1.1 degrees 
0 to 10 lbf 
0 to 24 inch-pound 


In figure 2 below, we see the annotated schematic of the Texas 9-string kinesthetic 
joystick. The two upper (vertical) planes of the joystick frame are constructed of clear acrylic in 
order to not obscure the operators’ vision as the robot work area is in front of the joystick and 
slightly to the right. 
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4. Implementation 

The handgrip position and orientation of the joystick is calculated in the dedicated 
microprocessor from the lengths of the 9 steel strings. The string lengths are measured by custom 
made rotary potentiometers. The microprocessor then maps the handgrip position and orientation to 
the end-effector of the robot. This mapping exists in software and can be scaled by the operator. 
Simultaneously, the robot end-effector load state is measured and mapped to the handgrip of the 9 
string joystick. 

A general transformation is used to map from the 6-dimensional force space of the robot 
end-effector to the 9-dimensional force space of the joystick. The 9-dimensional force space of the 
joystick is represented by the 9 independent servoed cable tensions, which can only act in tension, 
thus requiring the 3 constant forces of the compressive actuators in order to generate an arbitrary 
force at each connection of the X shaped hand-grip. Force Feedback is accomplished by holding 
the three air-cylinders at constant pressure and then controlling the tensions in the strings via 
current controlled servo-motors. As the problem of determining the cable tensions is 
underconstrained, an optimization has been performed to minimize the sum of the squares of the 
cable tensions. In mathematical form this is known as the pseudo-inverse of a non- square matrix. 

The dedicated microprocessor is a DEC Microvax II. The Microvax II computer represents 
the computational base for the application of the transformations, filtering, communications, and 
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control activities present in this system. With a sufficiently fast protocol for communications to and 
from the robot controller, the Microvax II would represent real-time computing power for this level 
of task. 


Each DC motor- transducer unit consists of a high-resolution .1% linearity rotary 
potentiometer attached to a spool wound with a steel cable. As the cable or 'string' is unwound 
from the spool, its length is proportional to the potentiometer resistance. Analog voltage 
measurement across the potentiometer can then be calibrated to the string length. The voltage 
readings from the transducers are continuous values which are converted via the DEC ADV1 1-C 
analog-to-digital converter board to digital information for the computer. Each transducer is driven 
by a brushless DC servo-motor to control the tension in the string. The Harowe motors are DC 
permanent magnet and brushless servo-motors with a stall torque rating of 35 pound-in. 

The force feedback control signals from the Microvax are converted by the DEC AAV 1 1 -C 
digital-to-analog converter to the continuous voltage signals needed for the Benton SC- 10 servo 
controllers. The servo-controllers operate in a current regulating manner to drive the DC motors. 
The use of a current control scheme over that of a voltage controlled one is critical to the 
performance of the 9 string joystick. The motor torque is proportional to the applied current. If the 
motors are powered in a voltage controlled mode then a back EMF forms which reduces the motor 
armature current. This results in the reduction of motor torque due to the circuit dynamics. This 
effect is equivalent to a mechanical damping. The magnitude of system dynamics is large enough to 
interfere with the operator's sensing of force feedback. In figure 3 below, the complete U.T. 
telerobotic testbed is shown. 



Figure 3: U.T. Telerobotic Test Facility 

The three air cylinders represent the prismatic joints in the legs of the Stewart-platform 
parallel mechanism. The compressive actuators are Benton B-120 single ended, 18 inch stroke air 
cylinders. Each air cylinder is supported by an adjustable bracket to the center of a 2 DOF gimbal 
or hook’s joint. The end of each air cylinder shaft is connected to a 3 DOF spherical joint. 
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composed of a steel universal joint with a ball bearing at each end. One spherical joint is connected 
to the end of each one of the three arms of the T shaped aluminum handgrip. At the connection of 
each spherical joint to an air cylinder shaft, three steel cables are attached. The intersection of the 
three strings with the air cylinder axis represents the point where the force at that arm of the 
handgrip is generated. 

Consequently, an arbitrary force vector (magnitude bounded) can be applied to each arm of 
the T shaped handgrip. Each force vector is limited by the applied maximum string tension. The 
three triad force vectors sum to produce the desired force and torque state at the center of the 
joystick grip. 

Software has been provided by the manufacturer to interface the Microvax II to the 
controller of the Cincinnati-Milacron T 3 - 726 . The Cincinnati-Milacron Inc. (CMI) host software is 
responsible for a time lag in the communications rate. The CMI software uses a non-real time 
protocol system known as DDCMP. 

The load state at the robot end-effector is sensed by a commercially available force/torque 
sensor. The sensor is a Lord corporation model 15/50 load cell. The model 15/50 is mounted to the 
wrist of the robot, and a connection is provided to affix a Telerobotics International EP 100/30 
robot gripper. The force sensor and the robot gripper are both driven by software implemented on 
the Microvax II. The robot end-effector is utilized by the telerobotic system operator via an on-off 
control button box. The button box is small and designed to be held in one hand by the operator to 
control the robot end-effector, while the other hand is in bilateral control with the robot arm. 

The fully integrated telerobotic system is represented in figure 4 by a signal flow chart. 
After the system undergoes the startup procedures the T 3-726 is placed in a remote mode in which 
the Microvax computer becomes a peripheral to the robot's controller. The operator then controls 
the system at two levels. In the first level, he must enter instructions into a menu-driven interactive 
routine on the computer terminal. At this stage, the operator can determine which control mode is 
desired. The different options available include; position-only control, resolved motion rate 
control, and kinesthetic control. In addition, the operator has the ability via the menu-driven 
terminal display to modify the spatial correspondence between the robot and the 9-string joystick. 
The operator can rereference the fixed joystick workspace to a new region of the robots 
workspace, he can scale the position and force mappings between the robot and the joystick either 
up or down from unity, and he can perform a smoothing operation on the position data to remove 
jitter from the robot’s motion. 

In the second level, the operator has placed the system software into control mode. The 
telerobotic system is then active. The operator by moving the handgrip within the limitations of the 
joystick workspace performs either a proportional move or sets a proportional velocity into effect 
for the robot end-effector. If the kinesthetic control loop is active, when the wrist of the robot is 
loaded by forces and torques, a scaled equivalent force and torque state at the operator's hand is 
generated. 

Testing of the DC motors has shown that a stall force greater than 12 pounds for several 
minutes yields high motor temperatures and declining performance. Therefore, the system is 
operated in the kinesthetic mode with a maximum string force of 12 pounds. From our design 
optimization procedure we calculated the pnuematic system set point and the maximum available 
force reflection for each triad of the joystick. The air cylinder constant force has then been set to 
IFI=14.83 pounds or 12.06 psig. The result is a maximum force feedback signal of 3.25 pounds at 
each arm of the T shaped handgrip without affecting the isotropic nature of the force reflection. 
This corresponds to a range of force/torque feedback for the handgrip from a pure maximum force 
capability of 9.75 pounds in any direction, to a pure maximum torque capability of 43 in-pounds 
about any axis. 
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Signal Flow Chart For 
Universal Force Reflecting Joystick 



Figure 4: Testbed System Communications 


5. Conclusions 

Current methods of control use limited, corrupted, or inappropriately coded information to 
the human operator as well as hardware and software of insufficient power, generality, and 
dexterity to exploit the full capacity of telepresence. 

The uniqueness of the 9 string joystick's geometry, the portability of its software, and the 
kinesthetic attractiveness of its operation make this man-machine interface a break with past 
engineering work in hand controllers and an excellent analysis tool for R&D. 

The Texas telerobotics testbed after completion has been evaluated and found to be 
functional, yet showing significant detractions. Indicating the importance and difficulty of 
achieving real-time 

telepresence in telemanipulation. The most crucial detraction to performance is the existence of a 
high level of coulomb friction in the joystick mechanism. The effect is concentrated in the sliding 
joint of the air cylinders. The implemention of a pnuematic system resulted in a masked force 
feedback, which blocks the joystick's transparency to bilateral communication flow. The friction 
force also had the effect of making small precise motions difficult. 

The high level of friction force in the pnuematic system also had the effect of obscuring the 
importance of friction from the motor-transducer units, and the inertial forces incurred in moving 
the joystick. A number of alternatives to a passive pnuematic system were considered such as 
motorized capstan, linear induction motor, and a linear mechanism employing a constant force 
spring. Another significant limitation to system performance is the slow update rate, or system 
cycle time. The protocol that allows information from the T3-726 controller to be sent to, or 
received from the Microvax II computer is not sufficiently fast to fulfill our design goal of 
achieving a 30 Hz run-time bilateral mapping. 
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The resulting update rate of 9-10 Hz represents a significant reduction in performance. The 
operator becomes cognitive of this time delay during precise positioning. Also, the time delay 
produces increasingly jerky motion in the manipulator as the distance between subsequent 
sampling points grows. 

In its present form the Texas 9-string kinesthetic joystick represents a proof-of-concept for 
a universal, parallel 6 DOF force reflecting manual controller for telerobotics. It does not yet 
achieve the demanding characteristics of transparency to information flow, and the system does not 
yet achieve the goal of telepresence. Currently, it is not expected to pursue improvement in the 
Texas 9- string joystick; but rather to use it for research into the issues necessary to design the next 
generation of man-machine interfaces. Current thinking for next generation interfaces include 
advanced hand-controllers based on redundant and hybrid (parallel and serial mechanical 
architecture) design as discussed by Sklar in [6]. 

Primarily, the research use for the Texas 9-string joystick is in such areas as human factors 
engineering. Results from that work would then push the design of man-machine interfaces based 
on a quantified understanding of issues such as joystick inertia, friction, cycle time, work volume, 
etc. 
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